1. Field of the Invention
This invention relates to a video image processing system.
2. Description of the Related Art
In a video image processing system video image fields or frames to be processed are represented by digital image data defining a multiplicity of picture elements (pixels) which together form the field or frame. In a 625 line or similarly sized video system each video frame typically requires about. 800K Bytes of data to represent all of the pixels forming the frame. Often image processing is applied to selected frames of a video clip comprising multiple frames and in order to facilitate the processing of the clip it is desirable to make available the data representing each frame on a frame random access basis. That is to say the time taken to access data representing any frame in the clip is substantially the same as the time taken to access the data for any other frame in the clip.
Frame random access storage devices are known. One such device comprises multiple disc stores arranged so that together they are able to receive and to output data at video display rate (e.g. 25 or 30 frames per second). Video stores are now available which comprise several disc storage devices connected together in parallel so as to enable several minutes of video to be stored on a frame random access basis. However, the cost of such store devices is relatively high and can be prohibitively expensive.
It is desirable that storage of data representing image sequences should be operated with a fixed file size, that is to say, with a store in which equal capacity is allocated for storage of the data relating to each image of the sequence. Where the stored image data is uncompressed, as is normally the case in an on-line system, such a store is the natural form to be adopted. However, the use of a store of fixed file size in an off-line system, where the image data is compressed before being stored, presents difficulties because the image data compression will result in files of data of different sizes for different images.
In an off-line system the number of video frames storable in a store of a given capacity can be increased (and the amount of storage space required can be reduced) by way of compression of the video data prior to storage. Data compression techniques are now sufficiently advanced to enable video data to be compressed with only small and acceptable losses in the quality of the image when the data is subsequently decompressed.
Generally, when a compression technique is applied to video data, the resulting compressed data bears no visual relation to the original image in the sense that if the compressed data was output directly to a monitor, the resulting image that would be displayed on the monitor would appear nothing like the original image. Indeed, the resulting image would probably be visually meaningless. Thus, whilst the original video data represents multiple pixels which together define an image frame or field, the compressed data corresponding thereto can be regarded as a block of data (referred to hereinafter as a file of data) having a known mathematical relationship to the data in the frame of data.
The amount of data compression that can be achieved depends on characteristics of the image. For example for a given compression, a frame of data representing a noisy picture such as a crowd scene will result in a much larger file of data than will a frame of data representing a quiet picture such as a screen of uniform colour. Consider the simple example of data compression by run length encoding wherein an image is defined by identifying groups of pixels in scan order having the same colour value (say 12 red pixels, followed by 7 green pixels, followed by 9 brown pixels and so on). In the case of a frame of data representing a uniform colour, that data can be reduced to a file of data comprising just a few bytes which define a group of approximately 800 thousand pixels (i.e. all pixels in the image) as having a given colour value. However, in the case of a frame of data representing a crowd scene the colour value of adjacent pixels can vary considerably and in the extreme a run length encoding of the frame of data would produce a file in which the colour of each pixel is individually defined, the file therefore containing more bytes of data than the original frame of data.
Run length encoding is a crude compression technique but more sophisticated approaches are available for use in the compression of video data, such as so-called JPEG compression and fractal compression. It is preferable to use these more sophisticated techniques in the compression of data representing video images because when the data is subsequently decompressed there is in many instances little discernable loss in the quality of the image represented thereby. However, these techniques too are sensitive to the characteristics (noisiness) of the picture. Consequently, compression of a frame of data representing a quiet picture will result in a smaller file of data than will the same compression of a frame of data representing a noisy picture. Consequently, when the degree of compression is fixed, compression of the frames of data in a video sequence will result in a series of files of data of varying size.
The storage of files of variable size in a random access store is not something that can easily be done efficiently. One way in which files of variable size may be stored randomly is by determining the largest possible size of file that will be produced from a frame of data representing the noisiest of images and allocating that amount of storage space to each file that is produced by the compression regardless of the actual size of the file. In the majority of cases video images will be less noisy than the worst case and the corresponding file of data will be smaller than the amount of space allocated to it. Consequently, not all of the storage space will be utilised in the storage of files of data, and this is wasteful of storage space. Where storage space is at a premium the degree of compression can be increased so that less space is required and need be allocated to each file of data. However, this approach results in unnecessarily high compression of frames of data representing quiet images and this in turn can result in the introduction of unwanted and visible artifacts in the quiet image represented by the subsequently decompressed frame of data.
The degree of compression applied to a frame of image data representing a given image to produce a file of data representing the image data in compressed form may be defined as a compression factor. For a given image, the greater the compression factor the more compressed is the data and the smaller therefore is the required file size. Thus, for a particular compression factor, if the image is a simple image, i.e. relatively quiet, it will require a smaller storage capacity than would be the case if the image were a complex image, i.e. relatively noisy. It should be appreciated from this that the term "compression factor" does not in fact relate to the ratio between the amount of data in a frame and the amount of data in the corresponding file resulting from the compression since the amount of data in the file will depend on the noisiness of the image as well as depending on the manner in which the data is compressed. Correctly, the expression "compression factor" refers to the amount of work that is done by the compression technique in compressing a frame of image data to produce a file of data, which work produces smaller files for relatively quiet images and larger files for relatively noisy images.
Thus, for a sequence of video images for example of a moving scene a range of different file sizes would be called for to store the sequence for a given degree of compression. As mentioned previously herein the storing of files of different sizes is an unattractive option because a uniform file size store considerably simplifies the filing structure of a machine where the permanent store is a disc store for which random access is provided and which can record to and delete from the store individual images of video. The use of a fixed file store also maintains the family similarity with machines in which the stored images are not compressed.